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Maximum  Likelihood  and  Bayesian  Parameter  Estimation  in  Item  Response  Theory 


Frederic  M.  Lord 

Abstract 

Advantages  and  disadvantages  of  joint  maximum  likelihood,  marginal 
maximum  likelihood,  and  Bayesian  methods  of  parameter  estimation  in  item 
response  theory  are  discussed  and  compared. 


Maximum  Likelihood  and  Bayesian  Parameter  Estimation  in  Item  Response  Theory* 


There  are  currently  three  main  approaches  to  parameter  estimation  in 
item  response  theory  (IRT): 

1.  Joint  maximum  likelihood,  exemplified  by  LOGIST,  yielding  maximum 
likelihood  estimates  (Wingersky,  1983). 

2.  Marginal  maximum  likelihood,  exemplified  by  B1L0G.  This  approach 
currently  yields  maximum  likelihood  estimates  of  item  parameters . 
Ability  parameters  can  be  estimated  subsequently,  using  Bayesian 
procedures  (Mislevy  &  Bock,  1981). 

3.  Bayesian  approaches:  parameter  estimates  are  usually  the  mode  (or 
mean)  of  the  posterior  distribution  of  the  parameter  estimated 
(Swaminathan  &  Gifford,  in  press). 

The  quantity  maximized  by  each  approach  is  shown  below  for  a  test  of  n 
items  administered  to  h  examinees.  P. (0  )  is  the  probability  of  success  on 

1  3 

item  i  for  examinee  a  at  ability  level  0  ,  Q.  (0  )  =  1  -  P.  (0  )  ,  u, 

a  i  a  l  a  ia 

is  the  response  of  examinee  a  to  item  i  ,  assumed  here  to  be  either  0  or  1, 

and  g(  )  denotes  a  prior  distribution  of  parameters. 

Joint  maximum  likelihood: 

h  n  u.  1-u. 

Maximize  L(0;a,b,c)  -II  II  [P.  (0  )]  ia  [0.  (0  )]  13  (1) 

»  -  ~  -  ,  .  .  l  a  la 

a-1  i=l 

N  n 

or  log  L(0 ;a,b,c)  -  I  l  (u  log  P  (0.  )  +  (1  -  u  )  log  Q. (0  )1 

•  *  *  1  o  xd  la  1  a 
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Marginal  maximum  likelihood  of  item  parameters ; 


N  “ 

Maximize  L(a,b,c)  =  II  /  g(8  )  L(9  ;a,b,c)  d8 

•  «  ,  8i  d  «  «  «  3 

a=  1 


Bayesian  modal  estimation: 


(2) 


Maximize  f(8;a,b,c)  =  L(8 ;a,b,c)g1(8)g2(a,b,c) 


(3) 


or  log  f (8 ;a,b,c)  =  log  L(8;a,b,c)  +  log  gj (6 )  +  log  g2(a,b,c) 


LOGIST  finds  the  ability  and  item  parameter  values  that  maximize  the 
likelihood  function  of  the  observations.  Bayesian  methods  typically  multiply 
this  likelihood  by  a  prior  for  each  of  the  parameters,  obtaining  the  joint 
posterior  distribution  of  the  parameters,  which  are  usually  assumed  to  be 
independently  distributed.  The  Bayesian  modal  estimates  (BME)  of  all  the 
parameters  are  the  values  at  the  mode  of  this  joint  posterior  distribution. 
Marginal  maximum  likelihood  multiplies  the  original  likelihood  by  a  prior  on 
ability,  eliminates  the  ability  parameters  by  integration,  and  obtains 
maximum  likelihood  estimates  of  the  item  parameters  by  maximizing  the 
resulting  'marginal'  likelihood  function.  Supplementary  Bayesian  procedures 
may  be  used  to  obtain  ability  parameter  estimates.  Bayesian  priors  on  item 
parameters  may  also  be  used  in  MMLE. 


When  approximately  parallel  test  forms  are  administered  year  after  year 
to  similar  populations  of  examinees,  it  becomes  possible  to  deduce  appropriate 
prior  distributions  for  the  item  and  the  ability  parameters  from  past  results. 
In  such  a  situation,  Bayesian  procedures  should  certainly  yield  better 
parameter  estimates  than  maximum  likelihood,  since  Bayesian  procedures  make 
use  of  more  information.  Even  in  the  absence  of  data  from  previous 
administrations,  Bock’s  BILOG  is  able  to  work  with  a  reasonable  prior 
distribution  of  ability  generated  directly  just  from  the  current  data. 

Marginal  maximum  likelihood  has  an  important  advantage  over  joint  maximum 
likelihood,  since  it  can  estimate  item  parameters  without  having  to  estimate 
ability  parameters.  The  advantage  is  not  one  of  computational  speed  but 
rather  of  theoretical  accuracy.  When  there  are  one  or  two  thousand  examinees 
and  40  or  more  items  per  person,  there  will  be  a  little  difference  between 
the  estimates.  In  cases  where  there  are  only  10  or  15  items  per  person, 
joint  maximum  likelihood  will  obtain  biased  estimates  of  ability  parameters, 
especially  at  low  ability  levels.  This  then  causes  the  item  parameters  to  be 
misestimated,  even  though  the  number  of  examinees  per  item  is  large. 

Let  us  turn  now  to  Bayesian  procedures.  From  the  mathematical 
statistician's  point  of  view,  one  clear  virtue  of  Bayesian  methods  is  that  if 
the  posterior  mean  is  used  as  a  parameter  estimate,  this  estimator  minimizes 
the  overall  mean  squared  error  of  estimation,  provided  the  appropriate  prior 
distribution  is  used.  In  the  case  of  ability  parameters,  for  example,  the 


quantity  minimized  is 


where  0  is  an  estimate  of  0  and  £  denotes  expectation  over  all  examinees. 
The  posterior  mean  achieves  this  important  result  by  accepting  increased 
estimation  bias  in  return  for  reduced  MSE.  The  BME  does  not  minimize  this  MSE 
unless  the  mode  of  the  posterior  distribution  coincides  with  its  mean,  which  is 
not  the  case  in  IRT  estimation  problems.  Nevertheless,  the  BME  may  be  close  to 
the  posterior  mean. 

Why  does  the  posterior  mean  do  better  than  the  maximum  likelihood  estimate 
(MLE)  in  minimizing  MSE?  When  item  parameters  are  known,  the  MLE  of  ability 
assigned  to  a  given  response  pattern  must  always  be  the  same.  In  Bayesian 
methods,  however,  the  ability  estimate  assigned  to  a  given  response  pattern 
depends  on  the  characteristics  of  the  entire  group  analyzed.  It  is  this 
additional  flexibility  that  allows  Bayesian  methods  to  obtain  a  smaller  MSE. 

Figure  1  shows  the  bias,  estimated  by  asymptotic  formula  (7)  accurate 
through  terms  of  order  l/n  ,  for  the  BME  of  0  (dashed  curve)  based  on  a 
normal  prior  and  for  the  MLE  of  0  (solid  curve),  calculated  for  an  90-item  SAT 
Verbal  test  that  first  came  to  hand,  using  the  three-parameter  logistic  model. 
The  values  shown  in  the  figure  assume  the  item  parameters  to  be  known. 

The  MLE  and  the  BME  are  biased  in  opposite  directions.  The  BME  is  more 
biased  than  the  MLE.  Note  that  neither  bias  is  linearly  related  to  0  ; 
consequently,  the  bias  cannot  be  corrected  by  a  simple  linear  transformation 
of  the  estimates. 

When  a  normal  prior  is  used  for  0  ,  the  asymptotic  standard  error  of  the 
BME  and  of  the  MLE  for  estimated  0  are  identical  to  the  usual  order  of 
approximation  (  l/n  ).  The  (familiar)  formula  for  both  asymptotic  standard 
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errors  is 
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S.E. (0 )  &  (  E 
i  =  l 
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v-1/2 


P.0. 

1  l 


(5) 


the  square  root  of  the  reciprocal  of  the  test  information  function  (I).  This 

is  also  the  asymptotic  formula  for  both  MSE's.  If  the  S.E.  were  written  out 

including  higher  order  terms,  the  Bayesian  S.E.  would  be  smaller  than  the 

2 

maximum  likelihood  S.E.  by  an  amount  of  order  1/n 

The  asymptotic  bias  in  the  MLE  in  the  three-parameter  logistic  model  is 
(Lord,  1983) 

D  n  1 

Bias (MLE(0 ) )  4“  l  a  I  ($  -  -5-  )  (6) 

r  i=l  c 


where  D 


1.7  ,  1.  = 

1 


P'2  3  P 

_ L_  p.  =  _ i 

PiQi  ’  1-39 


P.  -  c. 
_  1  1 

1  -  c. 


and  a^  and  c^  are  the  discrimination  parameter  and  lower  asymptote  for 


item  i  .  The  asymptotic  bias  for  the  BME  is  found  by  the  same  method  to  be 


Bias ( BME(0 ) )  &  bias(ML£(0))  -1  (7) 

Both  (6)  ana  (7)  are  of  order  1/n  . 

Because  of  the  bias  in  the  BME,  which  is  best  described  as  regression 
towards  the  mean,  the  variance  of  the  BME  across  examinees  is  less  than  the 


variance  of  the  true  0  values.  Many  people  apply  a  linear  transformation 
to  the  EME's  in  an  attempt  to  make  the  variance  (across  examinees)  of  the 
resulting  transformed  estimates  equal  to  the  variance  of  the  true  0  values. 
This  procedure  is  only  a  rough  approximation,  since  it  is  based  on  an 
assumption  of  linear  regression  of  BME  on  0  ,  whereas  the  true  regression 
is  curvilinear.  From  the  mathematical  statistician's  point  of  view, 
a  linearly  transformed  bME  or  posterior  mean,  or  a  curvilinearly  transformed 
BME  or  posterior  mean,  are  nonstandard  types  of  estimators.  Such  transform* 
estimators  no  longer  have  the  property  of  minimizing  MSE. 

A  further  problem  arises:  Minimizing  MSE  on  the  0  scale  is  not  the 
same  as  minimizing  hiSE  on  the  true-score  scale,  or  on  some  other  transformed 
ability  scale.  Bayesian  estimates  of  ability  will  differ  in  a  substantive 
way  depending  on  the  scale  chosen  for  measuring  ability.  This  problem  does 
not  arise  in  maximum  likelihood  estimation. 

Although  minimizing  MSE  on  the  0  scale  seems  appropriate  to  many  people, 
the  writer  believes  it  is  inappropriate.  Large  differences  in  0  ’s  at  the 
extremes  of  the  ability  scale  are  of  very  much  less  importance  for  most 
practical  purposes  than  smaller  differences  in  the  middle  of  the  scale.  If 
the  extremes  of  the  scale  were  important  to  us,  we  would  be  putting  more  easy 
items  or  more  hard  items  in  our  tests.  An  average  of  squared  differences, 
averaged  over  all  parts  of  the  scale,  is  thus  not  of  real  interest.  A 
procedure  that  attempts  to  minimize  such  an  average  will  devote  most  effort  to 
minimizing  the  large  squared  differences  found  at  the  extremes  of  the  scale, 
thus  partially  neglecting  more  important  differences  near  the  middle  of  the 


In  the  case  of  item-parameter  estimation,  the  idea  of  minimizing  the  MSE 
of  the  item  parameter  estimates  seems  inappropriate  for  a  different  reason. 

If  the  item  parameter  estimates  are  to  be  used  for  equating,  for  example,  the 
appropriate  quantity  to  minimize  is  the  squared  error  in  the  final  equating 
tables,  not  the  MSE  of  the  item  parameters.  If  the  items  are  to  be  used  for 
subsequent  adaptive  testing,  the  appropriate  criterion  is  a  mean  squared 
error  of  the  resulting  examinee  scores  on  the  adaptive  test. 

A  thought-provoking  circumstance  is  the  following:  Suppose  the  true  prior 
distribution  of  each  item  parameter  and  ability  parameter  were  known.  Given 
repeated  testings  over  a  few  years,  we  can  actually  come  close  to  this.  The 
leading  Bayesian  IRT  practitioners  prefer  not  to  use  such  a  'tight'  prior;  they 
prefer  to  use  a  more  diffuse  prior  that  produces  less  regression  of  the 
estimates  towards  the  mean.  This  attitude  derives  from  practical 
considerations  rather  than  from  Bayesian  logic. 

Use  of  Bayesian  priors,  even  diffuse  priors,  has  several  practical 
advantages  that  are  widely  appreciated: 

1.  Ability  estimates  (0  )  on  the  6  scale  are  automatically  restricted 
to  a  reasonable  range.  Infinite  estimates  do  not  occur. 

2.  Item  discrimination  parameter  estimates  never  try  to  become  infinite. 

3.  Estimated  lower  asymptotes  do  not  come  out  at  implausible  values, 
even  in  the  case  of  very  easy  items  that  provide  no  relevant  data 
for  estimating  the  asymptotes. 

The  last  two  advantages  convince  the  writer  that  Bayesian  priors  should 
probably  be  used  for  the  discrimination  and  the  lower-asymptote  parameters. 
Regression  towards  the  mean  in  estimates  of  these  parameters  has  less  serious 
implications  than  in  the  case  of  the  ability  and  difficulty  parameters. 


When  ability  parameter  estimates  are  regressed  towards  the  mean,  an 

A  A 

examinee's  score  (  0  or  some  transformation  of  9  )  depends  not  only  on  the 
examinee's  test  performance,  but  also  on  the  nature  of  the  entire  group  in 
which  he  or  she  happens  to  be  included.  If  the  group  as  a  whole  is  a  low- 
ability  group,  the  examinee's  score  may  be  regressed  downwards;  if  it  is  a 
high-ability  group,  the  examinee's  score  may  be  regressed  upwards.  If  the 
group  is  heterogeneous,  the  regression  effect  may  be  small;  if  the  group  is 
homogeneous,  the  regression  effect  could  be  large.  If  the  test  is  long  and 
reliable,  the  regression  of  scores  may  be  relatively  small;  if  the  test  is 
short  and  unreliable,  the  regression  effect  could  be  of  serious  concern. 

We  need  more  practical  experience  in  dealing  with  these  problems  in  real 
situations.  If  our  work  deals  with  a  single  test  and  a  single  group  of 
examinees,  regressed  ability  estimates  may  pose  no  problem,  because  the  rank 
order  of  the  examinees'  scores  will  be  little  affected.  If  our  work  deals  with 
comparisons  of  individuals  across  groups  and  across  tests,  with  data  analyses 
made  at  different  times,  we  may  want  more  experience  before  we  decide  exactly 
how  to  obtain  acceptable  results  for  large-scale  testing  programs.  Bayesian 
methods  may  be  the  ultimate  recourse,  but  we  need  considerable  experience  with 
them  before  we  can  be  sure  how  to  use  them  safely. 
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1  Nary  Schratz 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152 
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Navy 

1  Sr.  Robert  6.  Saith 
Office  of  Chief  of  Navel  Operations 
0P-987H 

Washington,  DC  20350 

l  Dr.  Alfred  F.  Saode 
Senior  Scientist 
Code  7B 

Naval  Training  Equipaent  Center 
Orlando,  FL  32813 

1  Dr.  Richard  Snot* 

Liaison  Scientist 
Office  of  Naval  Research 
Branch  Office,  London 
Box  39 

FPO  tie*  York,  NY  09510 

1  Dr.  Richard  Sorensen 
Navy  Personnel  KM)  Center 
San  Diego,  CA  92152 

1  Dr.  Frederick  Steinheiser 
CND  -  OF! 15 
Navy  Annex 
Arlington,  M  2C37C 

1  Nr.  Brad  Syepson 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Dr.  Frank  Vicino 
Navy  Personnel  RED  Center 
San  Diego,  CA  92152 

1  Dr.  Ronald  Weitzaan 
Naval  Postgraduate  School 
Departaent  of  Adainistrative 
Sciences 

Honterey,  CA  93940 

1  Dr.  Douglas  Wetzel 
Code  12 

Navy  Personnel  RM)  Center 
San  Diego,  CA  92152 

1  DR.  MARTIN  F.  NISK0FF 
NAVY  PERSONNEL  RE  D  CENTER 
SAN  DIE80,  CA  92152 

1  Nr  John  H.  Wolfe 
Navy  Personnel  RED  Center 
San  Diego,  CA  92152 


Navy 

1  Dr.  Wallace  Wulfeck,  111 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 
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Marine  Corps 

1  Jerry  Lehnus 
CAT  Project  Office 
HQ  Karine  Corps 
Kashi ngton  ,  DC  20380 

1  Headquarters,  U.  S.  Karine  Corps 
Code  NP1-20 
Washington,  DC  20380 

1  Special  Assistant  for  Karine 
Corps  Hatters 
Code  100K 

Office  of  Naval  Research 
800  N.  Quincy  St. 

Arlington,  VA  22217 

1  Kajcr  Frank  Yohannan,  USKC 
Headquarters,  Karine  Corps 
(Code  fIF  1-20) 

Washington,  DC  2O3E0 


Aray 

1  Dr.  Kent  Eaton 
Any  Research  Institute 
3001  Eisenhower  Blvd. 

Alexandria  ,  VA  22333 

1  Dr.  Clessen  Kartin 
Any  Research  Institute 
5001  Eisenhower  Blvd. 

Alexandria,  VA  22333 

1  Dr.  Killiai  E.  Nordbrock 
FHC-ADC0  Box  25 
APS,  NY  09710 

1  Dr.  Harold  F.  D’Meil,  Jr. 

Director,  Training  Research  Lab 
A rey  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Ccetander,  U.S.  Any  Research  Institute 
for  the  Behavioral  l  Social  Sciences 
ATTN:  PERI-BR  (Dr.  Judith  Orisanu) 

5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Kr.  Robert  Ross 

U.S.  Any  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Or.  Rotert  Saseor 
U.  S.  Any  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joyce  Shields 
Any  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
3001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Hilda  King 
Arty  Research  Institute 
5C01  Eisenhower  Ave. 

Alexandria,  VA  22333 
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Air  Force 

1  Or.  Ear!  A.  Alluisi 
HO,  AFHRL  iAFSC) 

Brocks  AFB,  TI  78235 

1  Hr.  Rayaond  E.  Christal 
AFNRL/HOE 

Brooks  AFB,  TI  78235 

1  Dr.  Alfred  R.  Fregly 
AFGSR/KL 

Boiling  AFB,  DC  20352 

1  Dr.  Patrick  Kyllonen 
AFHRL/NQE 

Brooks  AFB,  TI  78235 

1  Dr.  Randolph  Park 
AFHRL/NOAN 
Brooks  AFB,  TI  78235 

1  Dr.  Roger  Pennell 
Air  Force  Huaan  Resources  Laboratory 
Leery  AFB,  CC  BC230 

1  Dr.  Halcola  Ree 
AFHRL/MP 

Brooks  AFB,  TI  78235 


Departient  of  Defense 

12  Defense  Technical  Inforeation  Cencer 
Careron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Dr.  Clarence  McCoraick 
HG,  MEPCOM 
HEPCT-P 

2500  Green  Bay  Road 
Noprth  Chicago,  IL  60064 

1  Military  Assistant  for  Training  and 
Personnel  Technology 
Office  of  the  Under  Secretary  of  Defens 
for  Research  6  Engineering 
Root  3D129,  The  Pentagon 
Mashington,  DC  20301 

1  Dr.  N.  Steve  Sellian 
Office  of  the  Assistant  Secretary 
of  Defense  (RRA  6  L) 

2B269  The  Pentagon 
Mashington,  DC  20301 

1  Dr.  Robert  A.  Mistier 
OUSBRE  (ELS) 

The  Pentagon,  Rose  3D129 
Mashington,  DC  20301 


1  Major  John  Welsh 
AFHRL/M0AN 

Brooks  AFB  ,  TI  78223 
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Civilian  Agencies 

1  Dr.  Patricia  A.  Butler 
NIE-BRN  Bldg,  Stop  »  7 
1200  19th  St.,  NN 
Washington,  DC  20208 

1  Dr.  Vern  N.  Urry 
Personnel  RID  Center 
Office  of  Personnel  Hanageeent 
1900  E  Street  UK 
Washington,  DC  20415 

1  Dr.  Thoeas  A.  Mare 
U.  S.  Coast  Euard  Institute 
P.  0.  Substation  IS 
Oklahcoa  City,  OK  73169 

!  Dr.  Joseph  L.  Young,  Director 
Denary  V  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 


Private  Sector 

1  Dr.  Erlir.g  B.  Andersen 
Departsent  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 

1  Dr.  Isaac  Bejar 
Educational  Testing  Service 
Princeton,  N3  0B450 

1  Dr.  Denucha  Birenbaui 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Raaat  Aviv  69978 
Israel 

1  Dr.  Ke'P.er  Birke 
Personal staoaaat  der  Bundesseh'- 
D-5C00  Koeln  90 
NEST  6ERDANY 

1  Dr.  R.  Darrell  Bock 
Depa'tfent  of  Education 
University  of  Chicago 
Chicago,  11  60437 

1  Dr.  Arnold  Bohrer 
Section  of  Psychological  Research 
Caserne  Petits  Chateau 
CRS 

1009  Brussels 
Belgiut 

1  Dr.  Robert  Brennan 
Aeericar  College  Testing  Programs 
P.  0.  Box  16B 
Ioaa  City,  IA  52243 

1  Dr.  Blenn  Bryan 
620B  Pee  Road 
Eethesda,  MD  20817 

1  Dr.  Ernest  R,  Cadotte 
307  Stoke! y 

University  of  Tennessee 
Knoxville,  TN  37916 


1  Dr.  John  B.  Carroll 
409  Elliott  Rd. 
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Private  Sector 

1  Er.  Korun  Cliff 
Dept,  of  Psychology 
Uni*,  of  So.  California 
University  Per k 
Los  Angeles,  CA  90007 

1  Dr.  Hans  Croabag 
Education  Research  Center 
University  of  Leyden 
Boerhaavelaan  2 
2334  EH  Leyder. 

The  NETHERLANDS 

1  Lee  Crorbach 
16  Laburnuc  Road 
Atherton,  CA  9^205 

1  CTB/HcGrae-Hill  Library 
2300  Barden  Road 
Monterey,  CA  93940 

1  Hr.  Tisothy  Davey 
University  of  Illinhois 
Oepartaent  of  Educational  Psychology 
Urban, a,  1L  61801 

1  Er.  Dattpradad  Divgi 
Syracuse  University 
Departeent  of  Psychology 
Syracuse,  NE  33210 

1  Dr.  Eeeanuel  Donchin 
Departoent  of  Psychology 
University  of  Illinois 
Chaapaign,  1L  61820 

1  Dr.  Hei-Ki  Dong 
Ball  Foundation 
Rooe  314,  Building  B 
8C0  Roosevelt  Road 
Glen  Ellyn,  IL  60137 

1  Dr.  Eritz  D'asgoe 
Departeent  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 

Chaapaign,  IL  61820 

1  Dr.  Stephen  Dunbar 
Lindquist  Center  for  Heasureeent 
University  of  low 
lot* a  City,  !A  32242 


Private  Sector 

1  Dr.  John  H.  Eddins 
University  of  Illinois 
252  Engineering  Research  Laboratory 
103  South  Bathees  Street 
Urbana,  IL  61801 

1  Dr.  Susan  Eebertson 
PSYCHOLOGY  DEPARTMENT 
UNIVERSITY  OF  KANSAS 
Laurence,  KS  6604S 

1  ERIC  Facility-Acquisitions 
4633  Rugby  Avenue 
Bethesda,  HD  20014 

1  Dr.  Bsnjatir,  A.  Fairbarf,  Jr. 
Perforeance  Hetrics,  Inc. 

3825  Callaghan 
Suite  225 

San  Antonio,  TI  7822B 

1  Dr.  Leonard  Feldt 
Lindquist  Center  for  Heasuremt 
University  of  Iona 
Iowa  City,  1A  32242 

1  Uni  a.  Prof.  Dr.  6erhard  Fischer 
Liebiggasse  3/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Dor-ali  Fitzgerald 
University  of  Nee  England 
Aral  dale.  Nee  South  Kales  233! 
AUSTRALIA 

1  Dr.  Dexter  Fletcher 
University  of  Oregon 
Departeent  of  Computer  Science 
Eugene,  DR  97403 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  6  Neeian 
50  Houlton  Street 
Casbridge,  HA  02138 

1  Dr.  Janice  Gifford 
University  of  Hassachusetts 
School  of  Education 
Anherst,  HA  01002 
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Private  Sector 
1  Dr.  Robert  61a»er 

Learning  Research  t  Developeent  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
PITTSBURGH,  PA  15260 

1  Dr.  Harvin  D.  61 ock 
217  Stone  Hall 
Cornell  University 
Ithaca,  NY  14853 

1  Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  t  34th  Street 
Baltieore,  HD  21218 

1  DR.  JANES  6.  GREENO 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
397?  O’HARA  STREET 
PITTSBURGH,  FA  15213 

1  Dipl.  ?ad.  Michael  N.  Kabon 
Uni*ersitat  Dusseldorf 
Ernehungsaissenshaftliches  Inst.  II 
uni»ersitatsstr.  1 
D-4000  Dusseldo'f  I 
HES’  6ERHANY 

1  Dr.  Ron  Haafcleten 
School  of  Education 
University  of  Massachusetts 
Aeherst,  HA  01002 

1  P*-of.  Lot:  F.  Hornfce 
Universitat  Dussel dor f 
Emehungsaisser.schaftliches  Inst.  II 
Universitatsstr.  1 
Dusseldorf  1 

NEST  serkanv 

1  Dr.  Paul  Horst 
677  G  Street,  1184 
Chula  Vista,  CP  90010 

1  Dr.  Lloyd  Huaphre/s 
Department  o(  Psychology 
University  of  Illinois 
603  East  Daniel  Street 
Chaapaign,  IL  61820 


Private  Sector 

1  Dr.  Steven  Hunli 
Departeent  of  Education 
University  of  Alberta 
Edeonton,  Alberta 
CANADA 

1  Dr.  Jack  Hunter 
2122  Ccolidge  St. 

Lansing,  HI  48906 

1  Dr.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Coluabia,  SC  292GB 

1  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Laurence*; lie,  NJ  08148 

1  Profess:r  John  A.  Keats 
Department  of  Psychology 
The  University  of  Newcastle 
N.S.M.  2308 
AL'STRA.Ifi 

1  Dr.  Hi  Ilia*  Koch 
University  of  Texas-Austin 
Heasureaent  and  Evaluation  Center 
Austin,  TX  78703 

1  Dr.  Thosas  Leonard 
c/o  Dr.  Kelvin  R.  Novick 
Lindquist  Center  for  Heasureaent 
University  of  Icaa 
Icwa  City,  I A  52242 

1  Dr.  Alan  Lesgcld 
Learning  RAD  Center 
University  of  Pittsburgh 
3939  C’Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Richae!  Levine 
Departaent  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Chaapaign,  IL  6IB01 
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Private  Sector 

1  Dr.  Charles  Lewis 
Facsilteit  Socials  Wetenschappen 
R: jksuni versiteit  Groningen 
Oude  Poteringestraat  23 
9712GC  Broninqen 
Netherlands 

1  Dr.  Robert  Linn 
College  o<  Education 
University  of  Illinois 
Urbana,  IL  81801 

1  Dr.  Robert  Loci  can 
Center  For  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  N.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  Jaisa  Luasden 
Depart*ent  of  Psychology 
University  cf  Western  Australia 
Nediands  N.A.  4009 
AUSTRALIA 

1  Dr.  Gary  Narco 
Stop  31-E 

Educational  Testing  Service 
Pri"ceton,  NJ  08431 

1  Nr.  Robert  NcKmley 
Aten  can  College  Testing  Frograes 
P.0.  Bo<  168 
!o*a  City,  IA  32..4j 

1  Dr.  Barta'a  Neans 
Huian  PesOirces  Research  Qrganuatian 
300  North  Washington 
Alexandria,  VA  22314 

1  Dr.  Robert  Nislevy 
711  Illinois  Street 
Geneva,  IL  63134 

1  Dr.  N.  Alar  Nicewar.der 
University  of  OUahoea 
Departeent  of  Psychology 
OUahoea  City,  OK  73069 


Private  Sector 

1  Dr.  r.elvin  R.  Novick 
334  Lindquist  Center  for  Heasureent 
University  of  Iowa 
Iowa  City,  IA  52242 

1  Dr.  dates  Olson 
WICAT,  Inc. 

1875  South  State  Street 
0re»,  UT  B4057 

1  Wayne  H.  Patience 
Aeerican  Council  on  Education 
BED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  N* 

Washington,  DC  20034 

1  Sr.  Jaae;  Paulson 
Dept,  of  Psychology 
Portland  State  Unive3rsity 
P.0.  Bo*  751 
Portland,  OR  97207 

1  Dr.  Nark  D.  RecVase 
ACT 

P.  D.  Bo;-.  160 
Iowa  City,  IA  3224! 

1  Dr.  Lawerce  Rodner 
403  Eie  Avenue 
Taioea  Park,  ND  20012 

1  Dr.  J.  Ryan 
Departeent  of  Education 
University  of  South  Carolina 
Coluebia,  SC  292CB 

1  PROF.  cUN!K0  SANEJINA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE,  TN  37916 

1  Frani  L.  Scheldt 
Departeent  of  Psychology 
Bldg.  SG 

George  Washington  University 
Washington,  DC  20052 

1  Lowell  Schorr 
Psychological  6  Quantitative 
Foundat.ons 
College  of  Education 
University  of  Iowa 
Iowa  City,  IA  52242 
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Private  Sector 

1  Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuma-Kaigan 
Fujusawa  251 
JAPAN 

1  Dr.  Nilliaa  Sias 
Center  For  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria.  VA  22311 

1  Dr.  H.  Wallace  Sinaiko 
Proqraa  Director 

Manpower  Research  and  Advisory  Services 
Saithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

1  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08341 

1  Dr.  Feter  Sto’.off 
Center  For  Naval  Analysis 
200  North  PeaLrecard  Street 
Alexandria,  VA  22311 

;  Dr.  Hilliaa  Stout 
University  oF  Illinois 
Depart «nt  oF  Natheaatics 
Urbana,  It  61801 

1  Dr.  Hariharan  Suaainathan 
Laboratory  of  Fsychoietric  and 
Evaluation  Research 
School  oF  Education 
University  oF  Massachusetts 
Aeherst,  HA  01003 

1  Cr.  Kikuai  Tatsuoka 
Coeputer  Eased  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urbana,  It  61301 

1  Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Champaign,  It  61820 

1  Dr.  David  Thissen 
Department  oF  Psychology 
University  oF  Kansas 
Lawrence,  KS  66044 


Pr i vate  Sector 

1  Mr.  Sary  Thoeasson 
University  oF  Illinois 
Department  of  Educational  Psychology 
Champaign,  It  61820 

1  Dr.  Robert  Tsutakawa 
Departient  of  Statistics 
University  of  Missouri 
Columbia.  HO  65201 

1  Dr.  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
6C3  E.  Daniel  Street 
Champaign,  It  61820 

1  Dr.  V.  R.  R.  Uppuluri 
Union  Carbide  Corporation 
Nuclear  Division 
F.  C.  Box  Y 
Cat  Ridge,  TN  37330 

1  Dr.  David  Vale 
Assessment  Systeis  Co'-po'-ation 
223!  University  Avenue 
Suite  310 
St.  Paul,  HN  55114 

1  Dr.  Hcwaro  Nainer 
Division  o<  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08540 

1  Dr.  Ning-Mei  Nang 
Lindquist  Center  For  Heasureient 
University  of  Iowa 
Iowa  City  ,  IA  52242 

1  Dr.  Brian  Waters 
HumRRO 

IOC  North  Washington 
Alexandria,  VA  22314 

1  Dr.  David  J.  Weiss 
N66C  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Read 
Minneapolis,  HN  55455 

1  Or.  Rand  R.  Wilcox 
University  of  Southern  California 
Departient  of  Psychology 
Los  Angeles,  CA  90CC7 
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Private  Sector 

1  Eersan  Military  Representative 
ATTN:  Wolfgang  Nildegrube 
Strei tkr aef teait 
0-S300  Bonn  2 

4000  Brandywine  Street,  NW 
Washington  ,  DC  20016 

1  Dr.  Bruce  Williaas 
Beparttent  of  Educational  Psychology 
University  of  Illinois 
Urbana,  IL  61B01 

I  Ns.  Narilyn  Winge'Sky 
Educational  Testing  Service 
Princeton,  NJ  0B541 

1  Dr.  Beorge  Wong 
Biostatistics  Laboratory 
fleeorial  Sloan-Cettering  Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

1  Dr.  Wendy  Yen 
CTB/NcBraw  Hill 
Del  Norte  Research  Park 
Monterey,  CA  93940 
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